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Executive  Summary 

This  report  summarizes  the  findings  of  a  statistical  analysis  of  the  locations  of 
metallic  anomalies  detected  at  the  Pueblo  Precision  Bombing  Range  Number  2  in  Otero 
County,  Colorado,  and  at  the  Victorville  Precision  Bombing  Range  in  San  Bernardino 
County,  California.  The  purpose  of  the  study  was  to  explore  whether  statistical 
properties  of  the  pattern  of  anomaly  locations  can  be  used  to  discriminate  areas  likely  to 
contain  unexploded  ordnance  (UXO)  left  over  from  previous  bombing  practice  from 
those  unlikely  to  contain  UXO.  Techniques  for  discriminating  areas  with  and  without 
UXO  are  needed  because  historic  records  have  left  an  incomplete  account  of  previous 
military  training  activities,  so  that  locations  historically  used  for  target  practice  are  often 
unknown.  This  study  differs  from  previous  research  on  metallic  anomaly  data  at  former 
military  training  ranges  in  that  it  analyzes  the  spatial  pattern  of  the  discrete  locations  of 
the  anomalies,  rather  than  the  average  number  of  anomalies  per  unit  area.  The  results 
indicate  that  differences  in  spatial  pattern  may  be  a  distinguishing  feature  between  areas 
that  were  used  for  target  practice  and  those  that  are  unlikely  to  contain  UXO,  even  when 
a  large  number  of  ferrous  rocks  and  other  inert  metallic  anomalies  are  present.  We  found 
that  at  both  former  bombing  ranges,  the  anomaly  patterns  in  sample  areas  that  are  distant 
from  all  known  bombing  targets  are  consistent  with  a  complete  spatial  randomness  (CSR) 
pattern,  while  those  near  the  target  areas  fit  a  radially  symmetric,  bivariate  Gaussian 
pattern.  Furthermore,  anomaly  location  patterns  generated  by  surveys  with  airborne 
metal  detectors  have  the  same  statistical  properties  as  the  patterns  generated  by  surveys 
with  on-ground  detectors,  even  though  the  airborne  systems  detect  only  a  subset  of  the 
anomalies  found  by  the  ground-based  detectors.  Thus,  pattern  information  revealed  by 
airborne  surveys  with  metal  detectors  can  be  useful  in  identifying  areas  where  careful 
searches  for  UXO  are  needed. 
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Introduction  and  Objectives 


This  report  discusses  statistical  analyses  of  the  spatial  pattern  of  metallic 
anomalies,  buried  and  on  the  ground  surface,  detected  during  airborne  surveys  above  two 
former  Air  Force  bombing  ranges:  the  former  Pueblo  Precision  Bombing  Range  Number 
2  in  Otero  County,  Colorado,  and  the  Victorville  Precision  Bombing  Range  in  San 
Bernardino  County,  California.  The  main  purpose  of  the  analyses  is  to  determine 
whether  statistical  properties  of  anomaly  spatial  patterns  can  be  used  to  delineate  areas 
with  and  without  unexploded  ordnance  (UXO).  A  second  goal  is  to  estimate  the  expected 
number  of  UXO  at  given  locations  within  former  military  sites  based  on  the  results  of 
airborne  surveys.  Whereas  previous  statistical  characterization  of  metallic  anomaly  data 
at  UXO  sites  has  used  geostatistical  approaches  that  convert  discrete  anomaly  locations  to 
“concentrations”  of  UXO  per  unit  area  (McKenna  et  al.  2002;  McKenna  and  Saito  2003; 
Saito  et  al.  2005a, b),  this  paper  focuses  on  analytical  methods  that  preserve  the  discrete 
nature  of  the  pattern  information  that  the  data  contain. 

This  research  is  part  of  the  UXO  Wide  Area  Assessment  program  run  by  the 
Department  of  Defense  (DOD)  Environmental  Security  Technology  Certification 
Program  (ESTCP).  The  goal  of  this  program  is  to  provide  the  DOD  with  the  tools  needed 
to  fully  characterize  the  amount  of  land  in  the  United  States  that  is  contaminated  with 
UXO.  Due  to  the  long  time  period  during  which  the  military  has  conducted  live-fire 
training  in  some  areas  of  the  United  States,  combined  with  poor  record-keeping  over 
much  of  the  DOD’s  history,  the  amount  of  land  contaminated  with  UXO  and  specific 
geographic  boundaries  of  the  contamination  are  highly  uncertain  (Defense  Science  Board 
2003).  The  Defense  Science  Board  estimated  that  some  40,500  km2  (10  million  acres)  in 
the  United  States  need  to  be  surveyed  for  UXO,  based  on  information  on  past  military 
training,  but  that,  once  surveyed,  only  about  20%  of  that  total  land  is  likely  to  contain 
UXO  (Defense  Science  Board  2003).  The  Wide  Area  Assessment  Program  goal  is  to 
provide  technologies  that  can  be  used  to  survey  large  tracts  of  land  for  UXO  in  relatively 
short  time  periods  and  analytical  methods  that  can  be  used  to  assess  the  resulting  data  for 
evidence  as  to  which  areas  are  likely  to  and  unlikely  to  contain  UXO. 
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This  report  considers  whether  areas  without  UXO  have  patterns  of  metallic 
anomaly  locations  that  differ  from  the  patterns  in  areas  with  UXO.  It  assesses  whether 
statistical  properties  of  these  patterns  can  be  used  to  distinguish  areas  likely  to  contain 
UXO  from  areas  that  are  unlikely  to  contain  UXO.  The  report  first  provides  background 
information  about  wide-area  surveys  for  UXO  and  what  they  reveal.  It  then  discusses 
previous  approaches  to  statistical  analysis  of  the  data  from  such  surveys  and  how  the 
approach  in  this  paper  differs  from  this  previous  work.  Next,  it  provides  an  overview  of 
the  data  sets  used  as  the  basis  for  this  analysis.  Then,  it  analyzes  some  important 
statistical  properties  of  metallic  anomaly  locations  in  areas  unlikely  to  contain  UXO  and 
compares  them  to  areas  around  former  bombing  targets  at  both  ranges.  Last,  it  analyzes 
whether  patterns  revealed  by  airborne  sensors  are  consistent  with  those  observed  on  the 
ground. 


2  Background:  Wide-Area  Surveys  for  UXO 

Most  UXO  at  former  military  training  ranges  is  buried,  due  to  either  the  force  of 
the  initial  impact  or  to  later  soil  deposition.  Currently,  crews  charged  with  cleaning  up 
UXO  sites  must  rely  on  metal  detectors  to  locate  UXO.  They  traverse  the  ground 
equipped  with  either  hand-held  or  cart-mounted  detectors,  with  survey  times  limited  by 
the  walking  speed  of  the  operator,  terrain,  vegetation,  and  number  of  metallic  anomalies 
present.  As  is  well  known  in  the  UXO  community,  these  metal  detectors  generally  are 
unable  to  discriminate  between  buried  UXO  and  other  sources  of  metal,  such  as  ferrous 
rocks,  shrapnel,  hubcaps,  coins,  soup  cans,  and  other  sources  of  cultural  debris. 

Typically,  anywhere  from  10  to  1,000  metallic  items  are  excavated  for  every  UXO  found, 
depending  on  the  nature  of  the  site  (Defense  Science  Board  2003;  MacDonald  et  al. 

2004). 

Surveying  potentially  contaminated  land  with  hand-held  or  cart-mounted  metal 
detectors  is  a  slow  process.  Typically,  at  most  a  few  acres  can  be  searched  per  day 
(Defense  Science  Board  2003).  Over  the  past  several  years,  with  the  goal  of  speeding  up 
the  process,  organizations  funded  by  the  DOD  have  developed  airborne  systems  for  metal 


-8- 


detection.  Deployed  on  helicopters,  these  systems  automatically  record  the  geographic 
coordinates  of  the  metallic  signals  they  receive.  These  airborne  systems  can  survey 
hundreds  of  acres  per  day.  As  an  example,  at  the  Pueblo  range,  the  airborne  system 
averaged  456  acres  per  day  (Sky  Research  2006). 

A  limitation  of  the  airborne  platforms  is  that  the  signal  strength  of  metallic 
anomalies  decreases  at  a  rate  of  1/distance  ,  so  that  the  helicopters  must  fly  very  low  to 
the  ground  (1-2  m)  to  locate  anomalies  (Tuley  and  Dieguez  2005;  Sky  Research  2006). 
Even  when  maintaining  these  very  low  flight  altitudes,  the  airborne  systems  do  not  detect 
all  of  the  anomalies  found  by  on-ground  detectors,  which  can  be  held  within  millimeters 
of  the  soil  surface.  Nonetheless,  the  DOD  has  invested  in  continuing  to  develop  these 
airborne  systems  in  the  hopes  that  the  data  they  produce  will  be  useful  for  quickly 
identifying  the  areas  most  likely  to  contain  UXO,  thus  narrowing  the  scope  of  the 
problem  and  limiting  the  amount  of  searching  that  must  be  done  by  ground  crews. 

The  product  of  a  survey  for  UXO,  whether  conducted  via  helicopter-mounted  or 
on-ground  detectors,  is  a  map  showing  the  .^-coordinates  of  all  metallic  anomalies 
found.  Virtually  every  land  parcel  contains  some  buried  ferrous  material  (natural  and/or 
anthropogenic).  Thus,  the  detection  of  metallic  anomalies,  alone,  does  not  necessarily 
indicate  that  UXO  are  present.  In  fact,  the  number  of  metallic  anomalies  can  be  very 
high  in  areas  with  natural  deposits  of  ferrous  rock.  The  challenge  is  to  determine  how 
UXO-containing  areas  that  pose  risks  to  future  land  users  can  be  segregated  from  areas 
that  contain  inert  sources  of  metal. 


3  Previous  Approaches  to  Statistical  Analysis  of  Metallic  Anomaly  Data 

The  DOD  has  for  many  years  used  two  tools— “SiteStats/GridStats”  and  “UXO 
Calculator”— to  analyze  the  spatial  distribution  of  metallic  anomalies  at  potential  UXO 
sites  and  guide  site  sampling  (QuantiTech  1995;  Fanning  1999;  Engelhardt  et  al.  2000). 
Both  are  software  packages  that  field  crews  use  in  real  time,  on  lap-top  computers,  to 
estimate  the  density  of  UXO  items  (i.e.,  the  number  of  items  per  unit  of  land  area)  based 
on  metallic  anomaly  data.  However,  technical  reviewers  and  the  Environmental 
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Protection  Agency  (EPA)  have  criticized  the  statistical  model  that  underlies  these  tools, 
and  thus  DOD  has  sought  to  develop  improved  methods  (Fields  1999;  Engelhardt  et  al. 
2000).  Recent  research  funded  by  the  Strategic  Environmental  Research  and 
Development  Program  has  applied  geostatistical  kriging  methods  to  the  spatial  modeling 
of  anomalies  at  potential  UXO  sites  (McKenna  et  al.  2002;  McKenna  and  Saito  2003; 
Saito  et  al.  2005a,b). 


3.1  SiteStats/GridStats  and  UXO  Calculator:  complete  spatial  randomness  model 


SiteStats/GridStats  and  UXO  Calculator  both  assume  that  UXO  are  distributed  in 
two  dimensions  according  to  a  “complete  spatial  randomness”  (CSR)  model.  For  CSR  to 
hold,  the  spatial  arrangement  of  UXO  must  exhibit  two  properties  (Cressie  1993;  Diggle 
2003): 


1.  The  number  of  UXO  items  in  a  region  with  area  L4I  must  follow  a  Poisson 
distribution  with  mean  A1AI,  where  X  represents  the  average  number  of  UXO 
items  per  unit  area.  In  other  words,  the  probability  that  the  number  of  UXO 
items  in  any  area  of  size  IAI  equals  some  integer  k  is  given  by 


Pr(N  =  k) 


(X\A\)ke  A|/A| 


k\ 


k  =  0,  1,2,  3,. 


(1) 


2.  The  specific  locations  of  UXO  in  planar  region  A  are  an  independent  random 
sample  from  the  uniform  distribution  on  A. 


These  assumptions  imply  that  (1)  the  average  number  of  UXO  items  per  unit  area  is 
constant  across  the  site  and  (2)  any  given  location  on  the  site  has  an  equal  chance  as  any 
other  of  containing  UXO. 
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3.2  Limitations  of  the  CSR  model 


While  CSR  provides  a  realistic  model  of  some  spatial  processes,  its  use  for 
modeling  UXO  distributions  at  former  training  ranges  is  questionable  (Engelhardt  et  al. 
2000).  Since  soldiers  aim  their  weapons  at  targets,  it  is  logical  to  assume  that  UXO 
would  form  clusters  around  target  areas.  The  CSR  assumption,  in  contrast,  implies  that 
soldiers  fire  at  random,  without  regard  to  target  locations.  Thus  the  CSR  model  cannot 
capture  the  physical  reality  that  target  areas  will  contain  more  UXO  than  areas  that  are 
distant  from  targets. 

3.3  Geostatistical  methods  (kriging) 

Recent  research  has  sought  to  develop  a  stronger  foundation  for  UXO  spatial 
modeling  by  applying  techniques  from  geostatistics  (McKenna  2001;  McKenna  et  al. 
2003;  McKenna  and  Saito  2003;  Ostrouchov  et  al.  2003;  Saito  2005a,b).  Geostatistical 
methods  were  developed  initially  in  the  mining  industry  for  estimating  ore  grades  from 
observed  samples  and  are  now  used  in  a  variety  of  fields,  from  soil  science  to 
atmospheric  science  (Cressie  1993).  Their  primary  utility  is  for  predicting  the  value  of  a 
continuous  variable  (such  as  the  concentration  of  a  particular  contaminant  in  groundwater 
or  soil)  from  a  limited  number  of  environmental  samples  (Cressie  1993).  These 
approaches  make  use  of  the  spatial  correlation  in  the  variable  of  interest  that  may  occur 
for  locations  that  are  in  close  proximity.  For  example,  if  the  concentration  of  a 
groundwater  contaminant  is  high  in  one  location,  then  one  would  expect  relatively  high 
concentrations  in  nearby  locations,  rather  than  an  immediate  drop  from  high 
concentration  to  zero  concentration.  Estimates  of  the  value  of  the  variable  of  interest  at 
an  unsampled  location  are  determined  via  kriging,  in  which  values  of  the  variable  at 
sampled  locations  are  assigned  weights  that  reflect  their  spatial  correlation  to  the 
variable’s  value  at  the  unsampled  location. 

A  number  of  research  projects  have  applied  geostatistical  methods  to  extrapolate 
metallic  anomaly  information  from  limited  sampling  at  both  simulated  and  actual  UXO 
sites  (McKenna  2001;  McKenna  et  al.  2002;  McKenna  and  Saito  2003;  Saito  et  al. 
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2005a,b).  Because  these  methods  are  designed  for  analysis  of  continuous  random 
variables,  in  order  to  apply  them  to  UXO  sites,  the  discrete  points  specifying  potential 
UXO  locations  must  be  converted  to  a  continuous  random  variable.  Anomaly  pattern 
information  is  thus  converted  prior  to  geostatistical  analyses  to  “densities”  of  UXO  per 
unit  land  area:  the  sample  area  is  divided  into  cells  of  uniform  size,  the  number  of 
suspected  UXO  items  in  each  cell  is  counted,  and  this  number  is  divided  by  the  cell  area. 
Then,  this  density  value  is  assigned  to  the  spatial  coordinates  of  the  entire  cell.  The 
density  is  treated  as  if  it  varies  continuously  in  space,  so  that  kriging  can  be  employed. 

3.4  Limitations  of  kriging 

The  kriging  approach  is  useful  for  predicting  metallic  anomaly  density  on 
unsampled  parcels  of  land  when  only  a  limited  amount  of  the  land  can  be  investigated 
(Diggle  1976,  2003),  presuming  that  the  sampling  plan  is  sufficient  to  detect  areas  of  high 
anomaly  density.  However,  kriging  has  limitations  as  a  method  for  analyzing  wide-area 
survey  data.  The  primary  limitation  is  that  the  method  results  in  a  considerable  loss  of 
spatial  information.  This  lost  information  could  be  critical  in  distinguishing  UXO  signal 
patterns  from  signal  patterns  due  to  other  metal  sources,  such  as  metallic  rocks.  The 
geostatistical  approach  destroys  pattern  information  because  it  requires  the 
transformation  of  a  discrete  variable  (the  locations  of  potential  UXO  items)  to  a 
continuous  variable  (the  anomaly  density).  Cressie,  in  his  seminal  work  Statistics  for 
Spatial  Data,  identifies  this  problem: 

The  reduction  of  complex  point  patterns  to  counts  of  the  number  of  events 
in  random  quadrats  and  to  one-dimensional  indices  results  in  a 
considerable  loss  of  information.  Only  a  single  scale  of  pattern  can  be 
measured  by  a  sample  of  random  quadrats,  and  there  is  no  consideration  of 
...  the  relative  positions  of  events  within  quadrats,  so  most  of  the  spatial 
information  is  lost  (Cressie  1993). 
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The  information  on  locations  of  suspect  items  that  is  erased  when  geostatistical  methods 
are  employed  may  be  valuable  in  interpreting  spatial  patterns  that  could  provide  evidence 
for  the  presence  or  absence  of  UXO. 


4  Method 

4. 1  Spatial  point  pattern  analysis 

This  research  employs  methods  developed  for  the  analysis  of  spatial  point 
patterns.  Diggle,  in  one  of  the  definitive  texts  on  the  topic,  defines  a  spatial  point  pattern 
as  “a  set  of  locations,  irregularly  distributed  within  a  designated  region  and  presumed  to 
have  been  generated  by  some  form  of  stochastic  mechanism”  (Diggle  2003).  The 
locations  of  metallic  anomalies  detected  by  airborne  sensors  can  be  viewed  as  a  spatial 
point  pattern.  In  the  case  where  anomalies  are  UXO  at  a  former  firing  range,  the 
stochastic  mechanism  that  generated  the  pattern  was  the  launching  of  ammunition  at 
targets,  which  were  not  hit  with  perfect  accuracy.  Examples  of  other  kinds  of  spatial  point 
patterns  include  locations  of  trees  in  a  forest,  nests  in  a  bird  colony,  nuclei  in  a 
microscopic  tissue  sample,  and  stars  in  the  night  sky;  scientists  studying  all  of  these  kinds 
of  patterns  have  used  statistical  pattern  analysis  techniques  to  make  predictions  (Cressie 
1993;  Lawson  and  Densison  2002;  Diggle  2003).  Cressie  notes  that  point-pattern 
analysis  differs  from  geostatistics  in  that  “point  patterns  arise  when  the  important  variable 
to  be  analyzed  is  the  location  of  ‘events.’”  He  points  out  that  “geostatistical-type 
problems  are  distinguished  most  clearly  from  .  .  .  point-pattern-type  problems  by  the 
ability  of  the  spatial  index  ...  to  vary  continuously  over  a  subset  of  Rd”(Cressie  1993). 

Unlike  kriging,  spatial  point-pattern  analysis  preserves  all  information  about  the  locations 
of  the  events  of  interest. 

In  previous  research,  we  used  spatial  point-pattern  analysis  to  assess  anomaly 
distribution  patterns  at  two  former  Army  ground  artillery  ranges:  Fort  Ord,  California, 
and  Tobyhanna  State  Park,  Pennsylvania  (MacDonald  and  Small  2006).  We  concluded 
that  metallic  anomaly  locations  at  both  ranges  were  consistent  with  a  spatial  point- 
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process  model  known  as  the  Poisson  cluster  process.  This  model  accounts  for  the 
clustering  of  objects  of  interest  around  cluster  centers,  whose  locations  may  not  be  known 
in  advance.  Locations  of  firing  targets  changed  frequently  over  the  course  of  operation  of 
these  two  ranges,  and  data  on  target  locations  were  not  maintained.  Due  to  heavy 
vegetation  at  both  sites  and  the  long  time  periods  over  which  they  were  used,  target 
locations  could  not  be  determined  from  physical  evidence.  The  Poisson  cluster  process 
model  was  able  to  provide  statistically  significant  predictions  of  the  patterns  of  UXO 
locations  at  these  ranges,  despite  the  fact  that  target  locations  were  unknown,  because  it 
accounted  for  the  random  nature  of  target  placement,  as  well  as  for  clustering  of  UXO 
around  the  targets. 

4.2  Data  sources 

The  data  analyzed  in  this  paper  are  from  surveys  of  the  Pueblo  Precision  Bombing 
Range  Number  Two  and  the  Victorville  Precision  Bombing  Range.  The  data  were 
gathered  as  part  of  an  ongoing  ESTCP  demonstration  of  airborne  metal  detectors.  The 
Pueblo  range  served  as  a  U.S.  Army  Air  Forces  target  practice  site  between  1942  and 
1945  (U.  S.  Army  Corps  of  Engineers  1995).  Ammunition  dropped  on  this  range 
included  100-lb.  bombs  and  practice  bombs,  4-lb.  incendiary  bombs,  and  75-mm  cannon 
rounds  (Sky  Research  2006).  The  total  area  of  the  range  is  272  km"  (105  square  miles). 
The  Victorville  range  was  used  for  bombing  practice  during  the  same  time  period  as  the 
Pueblo  range.  The  historical  record  indicates  that  the  primary  (and  perhaps  only) 
munitions  used  were  100-lb.  practice  bombs  (Hathaway  et  al.  2007).  The  total  site 
acreage  is  approximately  22  km2  (8.6  square  miles). 

A  30-km"  land  parcel  at  the  Pueblo  range  has  been  designated  for  testing  wide- 
area  UXO  detection  technologies.  Within  this  demonstration  area,  a  swath  of  land  of 
about  3  km  x  9  km  was  surveyed  with  the  airborne  system.  Portions  of  this  area  also 
were  surveyed  with  an  array  of  metal  detectors  towed  on  the  ground  with  a  motorized 
vehicle.  Figure  1  shows  all  of  the  metallic  anomalies  detected  in  the  airborne  survey. 

Also  shown  are  the  locations  of  known  bombing  targets.  The  bombing  target  spatial 
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Figure  1  Locations  of  metallic  anomalies  identified  in  airborne  survey  of  the  study  area 
at  Pueblo  Precision  Bombing  Range  Number  Two,  Otero  County,  Colorado.  The  crosses 
show  locations  of  targets  at  which  pilots  aimed  during  bombing  practice. 


-  15  - 


coordinates  were  provided  by  ESTCP,  based  on  historical  information  and  aerial 
photography  (Sky  Research  2006). 

The  Pueblo  airborne  data  are  the  result  of  an  1 1-day  survey  conducted  by  Sky 
Research,  Inc.,  in  September  2005  (Sky  Research  2006).  The  metal  detection  system 
employed  in  this  survey  consisted  of  seven  Geometries  822  cesium  vapor  magnetometers 
mounted  on  a  9-m  Kevlar  boom  at  the  front  of  a  helicopter.  An  on-board  computer 
collected  the  metallic  signal  data  as  well  as  helicopter  location  coordinates  in  real  time. 
After  each  flight,  the  data  were  processed  to  remove  geologic  background  noise,  estimate 
the  spatial  coordinates  of  the  signals,  and  identify  which  anomalies  should  be 
investigated  as  possible  UXO.  Sky  Research  generated  the  list  of  anomalies  of  interest 
with  an  automated  signal-processing  routine  (Sky  Research  2006).  We  were  provided 
with  this  post-processed  data,  consisting  of  x,y  locations  of  the  anomalies  of  interest  as 
well  as  information  about  their  estimated  size,  depth,  orientation,  and  signal  strength. 

The  Victorville  airborne  data  also  were  gathered  by  Sky  Research,  Inc.,  using  the 
same  equipment  as  at  the  Pueblo  site.  Figure  2  shows  the  anomalies  detected  in  the 
airborne  surveys  and  the  location  of  Target  15,  which  is  the  focus  of  this  study.  Sky 
Research  used  an  automated  signal-processing  routine,  as  at  the  Pueblo  site,  to  identify 
the  locations  of  anomalies  of  interest  based  on  the  signals  from  the  airborne  detectors. 
The  data  set  provided  was  similar  to  that  provided  for  the  Pueblo  site. 


5  Results 

5.1  Spatial  patterns  of  anomalies  in  areas  with  and  without  UXO 

5.1.1  Pueblo  Range 

As  can  be  seen  from  Figure  1,  the  area  surveyed  by  helicopter  at  the  Pueblo  range 
included  land  that  is  more  than  2  km  on  all  sides  from  any  known  bombing  targets.  We 
chose  a  1,000-m  square  of  this  land  (labeled  in  Figure  1  as  “study  area”)  as  representative 
of  acreage  unlikely  to  contain  UXO.  Figure  3  shows  a  close-up  of  the  anomaly  locations 
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Figure  2  Locations  of  metallic  anomalies  identified  in  an  airborne  survey  of  the 
Victorville  Precision  Bombing  Range,  San  Bernardino  County,  California. 
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in  the  study  area.  We  compared  the  spatial  point  pattern  of  the  anomalies  in  this  area 
with  that  of  the  anomalies  surrounding  the  northern-most  target  (known  as  Target  3  and 
labeled  in  Figure  1  as  “north  target  study  area”).1  Figure  4  shows  a  detail  of  the 
anomalies  surrounding  this  target.  The  purpose  of  comparing  these  to  areas  was  to 
evaluate  the  hypothesis  that  the  metallic  anomaly  patterns  differ  in  areas  likely  and  not 
likely  to  contain  UXO  from  target  practice.  In  theory,  the  anomalies  in  target  areas 
should  form  a  pattern  that  clusters  around  the  target,  while  the  anomalies  in  areas  distant 
from  targets  should  not  be  clustered.  Although  the  anomaly  density  in  Figure  3  is  an 
order  of  magnitude  lower  than  that  in  Figure  4,  one  cannot  conclude  based  on  density 
alone  that  this  area  is  not  associated  with  a  former  bombing  target.  It  is  possible,  for 
example,  that  the  area  could  contain  UXO  from  a  target  on  which  fewer  bombs  were 
dropped  than  on  Target  3.  Likewise,  it  is  not  possible  to  determine  based  on  visual 
inspection  alone  whether  or  not  the  anomalies  shown  in  either  of  these  figures  form  a 
cluster  pattern.  Diggle  (2003)  observes,  “Intuitive  ideas  about  what  constitutes  a  random 
pattern  can  be  misleading.”  Statistical  analyses  are  needed  to  detect  true  cluster  patterns 
from  random  patterns  that  appear  to  have  clusters. 

The  spatial  point  patterns  of  the  two  sample  areas  can  be  compared  using  the  “ K 
function,”  which  is  commonly  used  to  summarize  information  about  spatial  point  patterns 
(Diggle  2003).  The  K  function  is  a  property  of  a  spatial  point  process  that  represents  how 
the  spatial  dependence  among  locations  of  events  varies  through  space  (Cressie  1993; 
Kaluzny  et  al.  1998;  Diggle  2003).  It  is  defined  for  specified  distance  h  as 


m 


(Expected  number  of  extra  events  within  distance  h  of  given  event) 


A, 


(2) 


The  sample  area  distant  from  known  targets  has  boundaries  616,100  m  <  X  <  617,100  m  and  4,172,500 
m  <  Y  <  4,173,500  m  (in  Universal  Transverse  Mercator  system  coordinates);  the  helicopter-mounted  detectors 
located  241  anomalies  in  this  area.  The  sample  area  around  the  northern  target  (Target  3)  has  boundaries 
616,666  m  <X<  617,666  m and 4,177,077  m <  Y < 4,178,077  m;  the  airborne  detectors  found  3,227  anomalies 
in  this  area. 
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Figure  4  Metallic  anomalies  located  during  airborne  survey  of  the  area  surrounding 
Pueblo  Target  3,  the  northernmost  target  in  Figure  1. 


-20- 


where  X  is  the  intensity,  which  is  defined  as  the  average  number  of  events  (in  this  case, 
metallic  anomalies)  per  unit  area.  For  the  CSR  model,  the  K  function  is  given  by 

k{h)=Kh2xX  =  nh2  (3 

X 

That  is,  the  expected  number  of  extra  events  increases  linearly  with  the  area  of  the 
circular  region  of  radius  h  around  a  particular  event.  K(h)  is  expected  to  be  larger  than 
for  the  CSR  model  (equation  3)  when  events  are  clustered  and  smaller  for  regularly 
spaced  events. 

A  common  method  of  evaluating  whether  a  given  type  of  parametric  model  fits  a 
spatial  point  pattern  is  to  estimate  K(h)  for  the  data  set  and  compare  it  with  K(h)  for  the 
model  (Cressie  1993;  Kaluzny  et  al.  1998;  Diggle  2003).  We  compared  the  anomaly 
patterns  in  the  two  sample  areas  to  the  CSR  model  using  K(h),  estimating  K(h)  from  the 
anomaly  location  data  as  described  in  Kaluzny  et  al.  (1998).  Figures  5  and  6  show  the 
results.  The  figures  show  the  mean  and  upper  and  lower  98%  confidence  intervals  for 
K(h)  for  the  CSR  model  and  K(h)  estimated  from  the  data  for  the  two  sample  areas. 
Figure  5  shows  that  the  CSR  model  of  K(h)  provides  a  virtual  perfect  fit  to  the  data  for 
the  area  unlikely  to  contain  UXO.  In  contrast,  Figure  6  shows  that  the  CSR  model  of 
K(h)  does  not  fit  the  estimated  K(h)  for  the  area  around  the  bombing  target.  In  this  latter 
sample  area,  the  values  of  K(h)  for  the  data  are  consistently  above  K(h)  for  the  CSR 
model,  showing  a  larger  number  of  points  around  any  given  data  point  than  would  be 
expected  under  CSR  and  thus  suggesting  clustering. 

We  also  compared  K-function  values  for  the  Pueblo  target  area  anomalies  with 
those  for  a  radially  symmetric,  bivariate  Gaussian  model  having  the  target  location 
(617,166  m;  4,177,577  m)  as  its  mean  and  a  standard  deviation  of  250  m.~  Our  previous 
analyses  of  data  from  the  two  Army  artillery  ranges  demonstrated  that  a  point-process 
model  that  assumes  UXO  items  are  distributed  around  targets  as  a  radially  symmetric 
Gaussian  distribution  fit  the  data  (MacDonald  and  Small  2006).  Analysis  of  the  Pueblo 


2  The  250-m  standard  deviation  was  estimated  from  the  mean  distances  from  the  northern  target  to  the 
anomalies  surrounding  it. 
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Figure  5  Evaluation  of  fit  of  the  CSR  model  to  the  anomaly  data  in  Figure  3  using  the 
function  K(h).  As  shown,  the  estimated  K(h)  for  the  anomaly  data  is  within  the  98% 
confidence  interval  for  the  CSR  model. 
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Figure  6  Evaluation  of  fit  of  the  CSR  model  to  the  anomaly  data  around  Pueblo  Target  3 
(Figure  4)  using  the  function  K(h).  As  shown,  the  estimated  K(h)  for  the  anomaly  data  is 
above  the  98%  confidence  interval  for  the  CSR  model,  indicating  that  the  anomalies  are 
clustered. 
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anomaly  data  indicates  that  this  assumption  also  appears  reasonable  for  the  target  area 
data.  Figure  7  shows  quantile-quantile  probability  plots  comparing  the  X  and  Y 
coordinates  of  the  anomalies  around  the  target  area  with  the  standard  normal  distribution. 
The  linearity  of  the  plots  shows  that  the  marginal  distributions  of  X  and  Y  are  consistent 
with  a  normal  distribution.  Further,  the  correlation  between  X  and  Y  is  statistically  not 
significantly  different  from  0  at  the  95%  confidence  level;  the  estimated  correlation 
coefficient  between  X  and  Y  is  0.03,  with  95%  confidence  interval  (0.00,  0.06).  It  is  thus 
reasonable  to  assume  that  a  Gaussian  model  with  zero  correlation  will  be  representative 
of  the  data. 

The  K  function  for  a  radially  symmetric  Gaussian  model  with  the  target  center  as 
its  mean  and  a  standard  deviation  of  250  m  matches  the  K  function  for  the  data  from  the 
target  area.  Figure  8  shows  this  result:  the  98%  confidence  interval  for  the  K  function 
for  the  Gaussian  distribution,  mean  value  for  this  K  function,  and  empirical  K  function  for 
the  data  are  nearly  indistinguishable.  The  bivariate  Gaussian  model  in  this  case  has 
probability  density  function 


f(x,y) 


1 

2;r(250)2 


exp 


(x-  61 7, 1 66)2  +  (y  -  4, 1 77, 577)2 " 
(250)2 


(4) 


In  this  case,  spatial  point-pattern  analysis  reveals  distinctly  different  spatial 
characteristics  of  anomaly  locations  in  the  area  distant  from  all  known  targets,  compared 
to  the  anomaly  locations  in  a  known  target  area.  The  anomaly  pattern  in  areas  distant 
from  targets  follows  the  CSR  model,  while  the  pattern  around  the  target  is  consistent  with 
a  bivariate,  radially  symmetric  Gaussian  model.  This  provides  evidence  that  spatial 
point-pattern  analysis  may  be  helpful  in  screening  areas  free  of  UXO  from  known  target 
areas. 

5.1.2  Victorville  range 

The  results  from  Victorville  lend  further  evidence  to  the  hypothesis  that  anomaly 
patterns  are  distinctly  different  in  areas  with  and  without  UXO.  The  Victorville  range 
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Figure  7  Normal-based  quantile-quantile  plots  of  the  x-  and  ^-coordinates  of  anomaly 
locations  in  the  target  areas  at  the  Pueblo  and  Victorville  ranges.  The  linearity  of  each 
plot  suggests  that  both  the  x-  and  y-coordinatcs  at  both  ranges  are  normally  distributed. 
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Figure  8  Evaluation  of  the  fit  of  a  radially  symmetric  Gaussian  model  to  the  anomaly  locations 
in  the  sample  area  around  Pueblo  Target  3  using  the  /^-function.  As  shown,  K(h)  estimated  for 
the  data  follows  the  expected  K(h)  for  the  Gaussian  model.  The  narrow  confidence  interval  for 
the  Gaussian  model  is  due  to  the  large  number  of  points  in  the  sample  area  around  Target  3  (see 
Figure  4). 
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contains  a  number  of  areas  with  high  anomaly  density  that  are  not  known  to  be  associated 
with  former  bombing  targets.  Our  analysis  focused  on  three  of  these  areas  (shown  on 
Figure  2),  and  on  the  anomalies  around  Target  15  (Figure  2).  In  each  case,  we  selected  a 
square  sample  area  of  1000  m  on  a  side.  We  conducted  the  same  type  of  analysis  as  for 
the  Pueblo  data  to  determine  whether  or  not  the  anomaly  patterns  in  these  areas  are 
consistent  with  a  CSR  distribution.  Figures  9-11  show  that  the  patterns  in  the  three  areas 
not  associated  with  the  target  fit  a  CSR  model,  even  though  these  areas  contain  high 
anomaly  densities.  Figure  12  shows  that  the  pattern  around  the  target  is  not  consistent 
with  a  CSR  pattern.  The  empirical  ^-function  values  are  consistently  above  the  98% 
confidence  interval  for  a  CSR  model  up  through  distances  of  about  600  m,  providing 
evidence  of  clustering. 

We  fitted  the  anomaly  locations  around  Target  15  to  a  bivariate  Gaussian  model 
using  the  same  process  as  for  the  Pueblo  data.  As  for  the  Pueblo  data,  the  X  and  Y 
coordinates  appear  on  a  quantile-quantile  plot  (Figure  7)  to  be  normally  distributed.  The 
correlation  between  X  and  Y  is  -0.04,  with  95%  confidence  interval  (-0.09,  0.006),  and  the 
standard  deviations  of  both  X  and  Y  are  147  m,  indicating  that  a  radially  symmetric 
Gaussian  model  is  consistent  with  the  anomaly  locations  around  the  target.  Figure  13 
compares  the  empirical  ^-function  for  the  anomalies  in  the  target  area  to  the  theoretical 
^-function  for  a  radially  symmetric  Gaussian  model  with  the  target  as  its  mean  and  a 
standard  deviation  of  147  m.  As  shown,  this  model  fits  the  data  well,  providing  further 
evidence  that  the  anomaly  pattern  around  the  target  is  a  cluster  pattern  centered  at  the 
target. 

5.2  Predicting  ground-based  survey  results  from  airborne  data 

Due  to  the  drop-off  of  magnetic  signal  strength  with  distance,  airborne 
magnetometers  detect  fewer  anomalies  than  ground-based  systems.  This  raises  two 
questions.  First,  are  the  patterns  revealed  by  airborne  surveys  consistent  with  those 
produced  by  ground  surveys?  Second,  how  can  the  number  of  anomalies  in  the  survey 
area  be  estimated,  given  the  limitations  in  detection  ability  of  the  airborne  system? 
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Figure  9  Evaluation  of  fit  of  the  CSR  model  to  anomaly  data  from  Area  1  of  the 
Victorville  range  using  the  function  K(h).  As  shown,  the  empirical  K(h)  for  the  anomaly 
data  is  within  the  98%  confidence  interval  for  the  CSR  model. 
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Figure  10  Evaluation  of  fit  of  the  CSR  model  to  anomaly  data  from  Area  2  of  the 
Victorville  range  using  the  function  K(h).  As  shown,  the  empirical  K(h)  for  the  anomaly 
data  is  within  the  98%  confidence  interval  for  the  CSR  model. 
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Figure  11  Evaluation  of  fit  of  the  CSR  model  to  anomaly  data  from  Area  3  of  the 
Victorville  range  using  the  function  K(h).  The  empirical  K(h)  for  the  anomaly  data  is 
within  the  98%  confidence  interval  for  the  CSR  model. 
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Figure  12  Evaluation  of  fit  of  the  CSR  model  to  anomaly  data  from  Target  15  of  the 
Victorville  range  using  the  function  K(h).  As  shown,  the  empirical  K(h)  for  the  anomaly 
data  is  above  the  98%  confidence  interval  for  the  CSR  model  for  distances  up  to  about 
600  m,  indicating  that  the  anomalies  are  clustered. 
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Figure  13  Evaluation  of  fit  of  a  radially  symmetric  Gaussian  model  to  anomaly  data 
from  Target  15  of  the  Victorville  range  using  the  function  K(h).  As  shown,  the  empirical 
K(h)  for  the  anomaly  data  is  within  the  98%  confidence  interval  for  the  Gaussian  model. 
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5.2.1  Airborne  versus  ground-based  anomaly  patterns 


To  assess  whether  the  anomaly  patterns  are  similar  for  the  data  sets  from  the 
airborne  and  ground-based  surveys,  we  compared  the  ^-function  values  for  the  data  sets 
in  areas  of  the  Pueblo  site  that  were  surveyed  with  both  types  of  sensor  platforms.  Figure 
14  shows  the  results  of  such  an  analysis  (for  a  sample  area  designated  as  3C  in  the  data 
set):  it  plots  K(h)  for  the  ground-based  data  against  K(h)  for  the  airborne  data.  The  result 
is  a  straight  line  with  slope  1.05.  This  provides  evidence  that  the  airborne  anomalies 
represent  a  thinned  subset  of  the  spatial  point-process  that  generated  the  ground-based 
anomalies.  The  results  for  the  other  three  areas  that  were  surveyed  with  both  airborne 
and  on-ground  detectors  were  similar.  The  implication  is  that  the  spatial  pattern  revealed 
by  the  airborne  survey  is  the  same  as  that  revealed  by  a  ground-based  survey.  Thus,  any 
inferences  made  about  the  spatial  point  pattern  observed  from  airborne  metal  detectors 
also  should  apply  to  the  anomaly  set  observed  by  ground-based  detectors,  with  the  air¬ 
generated  pattern  representing  a  thinned  subset  of  the  ground-generated  pattern. 

5.2.2  Airborne  versus  ground-based  estimates  of  expected  number  ofUXO 

Since  the  anomaly  data  from  airborne  surveys  provide  only  a  partial 
representation  of  the  set  of  anomalies  located  by  ground-based  surveys,  an  important 
question  is  how  many  anomalies  the  airborne  surveys  miss.  If  reasonable  estimates  for 
the  probability  that  an  airborne  system  will  detect  anomalies  resembling  UXO  can  be 
made,  then  airborne  survey  results  can  be  converted  directly  to  estimates  of  the  expected 
number  of  anomalies  in  any  given  region.  This  can  help  DOD  to  further  define  the  scope 
of  the  UXO  problem  and  develop  more  accurate  estimates  of  remediation  costs  based  on 
airborne  wide-area  surveys,  before  ground  surveys  are  completed. 

We  used  the  data  from  all  four  of  the  areas  that  were  fully  surveyed  by  both 
ground-based  and  airborne  magnetometers  at  the  Pueblo  range  to  assess  the  probability 
that  the  airborne  detectors  will  find  anomalies  located  by  ground-based  detectors.  Table 
1  and  Figure  15  summarize  probabilities  of  detection  for  the  four  sample  areas  for 
different  detection  radii.  We  define  the  detection  radius  as  the  distance  from  a  given 
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Figure  14  K(h)  for  the  ground-based  anomalies  in  Pueblo  Area  3C  compared  to  K(h)  for 
the  anomalies  identified  by  the  airborne  system.  As  shown,  the  K(h)  values  are  nearly 
equivalent.  This  provides  evidence  that  the  two  anomaly  patterns  are  drawn  from  the 
same  type  of  spatial  point-pattern  distribution.  The  anomalies  identified  by  the  airborne 
system  thus  may  represent  a  thinned  version  of  the  spatial  point-process  of  the  anomalies 
identified  by  the  ground-based  system. 
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Table  1  Effect  of  Search  Radius  on  Probability  of  Detection 


Search  Radius  (m) 
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0.292 

0.319 
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Figure  15  Probability  that  the  airborne  magnetometers  will  locate  anomalies  found  by  the 
ground-based  magnetometers  as  a  function  of  detection  radius.  Detection  radius  is  the 
distance  around  an  anomaly  found  by  the  ground-based  magnetometer  within  which  the 
airborne  detector  must  generate  a  signal  for  the  anomaly  to  have  been  counted  as  a 
detected  item. 
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ground  anomaly  to  the  corresponding  airborne  signal.  If  the  airborne  sensors  locate  an 
anomaly  within  the  given  detection  radius  of  an  anomaly  found  in  on-ground  surveys, 
then  we  credit  the  airborne  sensors  as  having  successfully  detected  that  anomaly.  As  can 
be  seen  in  Table  1  and  Figure  15,  the  detection  probability  increases  with  detection 
radius.  The  maximum  detection  probability  is  35.5%,  at  the  very  large  search  radius  of  4 
m.  For  a  more  realistic  detection  radius  of  1.5  m,  the  probability  of  detection  is  22-27%. 
As  a  point  of  comparison,  an  analysis  in  2005  of  anomaly  data  collected  at  a  former 
practice  bombing  range  used  by  Kirtland  Air  Force  Base  found  that,  with  a  detection 
radius  of  1.5  m,  a  helicopter-mounted  detection  system  found  fewer  than  25%  of  60-mm 
mortars  and  75%  of  105-mm  rounds;  on  average,  this  system  found  56%  of  UXOs 
emplaced  in  a  controlled  field  test  (Tuley  and  Dieguez  2005).  The  lower  detection 
probabilities  observed  in  our  research  may  be  a  result  of  differences  in  site  conditions, 
such  as  amount  of  geologic  interference,  or  in  differences  in  the  algorithms  used  to 
identify  which  signals  generated  by  the  airborne  system  represented  anomalies  worthy  of 
investigating.  In  the  Kirtland  Air  Force  Base  study,  geophysicists  hand-picked  anomalies 
for  investigation  based  on  the  signal  information,  while  in  the  data  we  used,  anomalies 
lists  were  generated  by  a  computer  search  algorithm. 


6  Conclusions 

Analysis  of  metallic  anomalies  detected  in  airborne  and  on-ground  surveys  of  the 
Pueblo  and  Victorville  bombing  ranges  suggest  the  following  conclusions: 

•  Differences  in  the  spatial  point  patterns  of  metallic  anomalies  may  be  a 

distinguishing  feature  between  areas  that  were  used  for  weapons  training  and 
areas  not  used  for  such  training.  At  both  bombing  ranges,  anomalies  in 
sample  areas  that  are  distant  from  all  known  bombing  targets  are  completely 
spatially  randomly  distributed,  while  those  near  the  targets  do  not  fit  this  CSR 
model. 
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•  A  symmetric,  bivariate  Gaussian  model  appears  to  represent  the  anomaly 
pattern  around  targets  at  both  the  Victorville  and  Pueblo  ranges.  This  model 
may  be  useful  in  identifying  how  far  from  the  target  searches  for  UXO  should 
be  conducted.  As  an  example,  95%  of  the  anomalies  are  expected  to  be  within 
two  standard  deviations  of  the  target  center,  while  99.7%  are  expected  to  lie 
within  three  standard  deviations  of  the  target  center. 

•  Based  on  analyses  of  the  Pueblo  data,  the  airborne  anomaly  locations  appear 
to  be  drawn  from  a  spatial  point  process  that  is  a  thinned  version  of  the 
process  that  generated  the  anomalies  detected  by  the  ground-based  system.  In 
other  words,  the  airborne  system  is  able  to  reveal  the  same  type  of  pattern 
information  as  the  ground-based  system. 

•  The  probability  that  helicopter-mounted  metal  detectors  will  locate  anomalies 
detected  by  ground-based  systems  depends  on  how  close  the  airborne  anomaly 
must  be  to  an  anomaly  found  by  the  ground-based  system  to  be  declared  a 
successful  detection.  For  a  detection  radius  of  1.5  m,  the  probability  of 
detection  of  the  areas  surveyed  in  the  Pueblo  site  ranged  from  22-27%. 

As  the  DOD  moves  forward  with  development  of  airborne  systems  for  UXO 
detection,  the  kinds  of  spatial  point-pattern  analysis  demonstrated  in  this  report  can  help 
to  document  the  geographic  boundaries  of  areas  likely  to  contain  UXO.  Where  the 
spatial  pattern  of  anomalies  is  not  consistent  with  that  expected  at  UXO  sites,  statistical 
analysis  of  the  pattern  information  can  provide  evidence  that  the  area  is  not  a  high- 
priority  for  exhaustive  ground-based  searches  for  UXO.  Where  the  pattern  analysis 
suggests  that  UXO  is  present,  pattern  information  can  be  used  to  predict  the  boundaries 
of  former  target  areas  and  thus  reduce  the  amount  of  acreage  that  must  be  searched. 
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